Image status determining method an apparatus, device, system, and computer storage medium

ABSTRACT

A method comprises obtaining a pathology image set using a microscope, the pathology image set including at least a to-be-evaluated image and one or more associated images, the associated images and the to-be-evaluated image are consecutive frame images acquired using the microscope. The method comprises determining a first status corresponding to the to-be-evaluated image according to the pathology image set, the first status being used for indicating a motion change of the to-be-evaluated image during the acquisition and the first status includes a plurality of predefined states. The method comprises in accordance with a determination that the first status corresponds to a static state of the plurality of predefined states, determining a second status corresponding to the to-be-evaluated image, the second status indicating a change in image clarity of the to-be-evaluated image. This application further discloses an image status determining apparatus, a device, and a computer storage medium.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation application of PCT Patent ApplicationNo. PCT/CN2020/090007, entitled “IMAGE STATE DETERMINATION METHOD ANDDEVICE, APPARATUS, SYSTEM, AND COMPUTER STORAGE MEDIUM” filed on May 13,2020, which claims priority to Chinese Patent Application No.201910457380.4, filed with the State Intellectual Property Office of thePeople's Republic of China on May 29, 2019, and entitled “IMAGE STATEDETERMINATION METHOD, DEVICE AND SYSTEM BASED ON PATHOLOGICAL IMAGE”,all of which are incorporated herein by reference in their entirety.

FIELD OF THE TECHNOLOGY

This application relates to the field of artificial intelligence (AI),and in particular, to a pathology image-based image status determiningmethod and apparatus, a device, a system, and a non-transitorycomputer-readable storage medium.

BACKGROUND OF THE DISCLOSURE

Pathological examination has been widely applied to clinical work andscientific research, and a main method for performing pathologicaldiagnosis by a doctor is to observe a slice and observe cell morphologyand tissue structure after enlarging the slice by 40 to 400 times, tomake the diagnosis. A smart microscope and a digital pathology scannerare the most common tools used by the doctor.

The smart microscope usually has a camera and may consecutively acquireimages for a field of view of the microscope. The acquired images areused for a plurality of microscope tasks such as an automatic imagesaving task and an image-based real-time artificial intelligence (AI)auxiliary diagnosis.

The camera of the smart microscope is generally a high-resolutionhigh-speed industrial camera with a high frame rate of image acquisition(up to several frames to dozens of frames of images per second).Moreover, the images each have a large volume (a total number of pixelsof a single image may be up to 4,000,000 or above), and a large amountof image data may be generated in a short time. If an image status canbe evaluated, images acquired by the smart microscope can be screenedbased on the image status, so that the processing efficiency of amicroscope task is improved. However, there is no solution fordetermining an image status in the related art.

SUMMARY

Embodiments of this application provide a pathology image-based imagestatus determining method and apparatus, a device, a system, and acomputer storage medium, which can perform moving state evaluation andimage clarity evaluation on an acquired image, so that image screeningmay be performed based on a determined image status, to adapt torequirements of different microscope tasks, thereby improving the taskprocessing efficiency.

In view of this, the embodiments of this application provide a pathologyimage-based image status determining method, including:

acquiring a pathology image set using a microscope, the pathology imageset including at least a to-be-evaluated image and one or moreassociated images, the associated images and the to-be-evaluated imagebeing consecutive frame images acquired using the microscope;

determining a first status corresponding to the to-be-evaluated imageaccording to the pathology image set, the first status being used forindicating a motion change of the to-be-evaluated image during theacquisition and the first status includes a plurality of predefinedstates; and

in accordance with a determination that the first status corresponds toa static state of the plurality of predefined states, determining asecond status corresponding to the to-be-evaluated image according tothe pathology image set, wherein the second status indicates a change inimage clarity of the to-be-evaluated image.

The embodiments of this application further provide an image statusdetermining apparatus, including:

an obtaining module, configured to obtain a pathology image set by usinga microscope, the pathology image set including at least ato-be-evaluated image and associated images, the associated images andthe to-be-evaluated image being consecutive frame images;

a determining module, configured to determine a first statuscorresponding to the to-be-evaluated image according to the pathologyimage set obtained by the obtaining module, the first status being usedfor indicating a motion change of the to-be-evaluated image; and

the determining module, further configured to determine, when the firststatus is a static state, a second status corresponding to theto-be-evaluated image according to the pathology image set, the secondstatus being used for indicating a change in image clarity of theto-be-evaluated image.

The embodiments of this application further provide a smart microscopesystem, including an image acquisition module, an image processing andanalyzing module, a pathological analysis module, a storage module, anda transmission module,

the image acquisition module being configured to obtain a pathologyimage set, the pathology image set including at least a to-be-evaluatedimage and associated images, the associated images and theto-be-evaluated image being consecutive frame images;

the image processing and analyzing module being configured to determinea first status corresponding to the to-be-evaluated image according tothe pathology image set, the first status being used for indicating amotion change of the to-be-evaluated image; and

determine, when the first status is a static state, a second statuscorresponding to the to-be-evaluated image according to the pathologyimage set, the second status being used for indicating a change in imageclarity of the to-be-evaluated image;

the storage module being configured to store the to-be-evaluated imagewhen the first status is of moving state-to-static state transition; and

store the to-be-evaluated image when the second status is of blurredstate-to-clear state transition;

the pathological analysis module being configured to performpathological analysis on the to-be-evaluated image when the first statusis of moving state-to-static state transition; and

perform pathological analysis on the to-be-evaluated image when thesecond status is of blurred state-to-clear state transition; and

the transmission module being configured to transmit the to-be-evaluatedimage when the first status is of moving state-to-static statetransition, or the first status is of static state-to-moving statetransition, or the first status is a moving state; and

transmitting the to-be-evaluated image when the second status is ofblurred state-to-clear state transition or the second status is of clearstate-to-blurred state transition.

The embodiments of this application further provide a terminal device(e.g., a computer device, an electronic device, etc.), including amemory and a processor,

the memory being configured to store a computer program; and

the processor being configured to perform the pathology image-basedimage status determining method provided in the embodiments of thisapplication when executing the computer program in the memory.

The embodiments of this application further provide a non-transitorycomputer readable storage medium, storing computer executableinstructions, and the computer executable instructions being configuredto perform the pathology image-based image status determining methodprovided in the embodiments of this application.

The application of the pathology image-based image status determiningmethod and apparatus, the device, the system, and the non-transitorycomputer readable storage medium provided in the embodiments of thisapplication has at least the following beneficial technical effects.

The embodiments of this application provide a pathology image-basedimage status determining method. First, a pathology image set isobtained, the pathology image set including at least a to-be-evaluatedimage and associated images, the associated image being a previous frameof image adjacent to the to-be-evaluated image, then a first statuscorresponding to the to-be-evaluated image is determined according tothe pathology image set, the first status being used for indicating amotion change of the to-be-evaluated image, and a second statuscorresponding to the to-be-evaluated image is determined according tothe pathology image set when the first status is a static state, thesecond status being used for indicating a change in image clarity of theto-be-evaluated image. In the foregoing manner, moving state evaluationand image clarity evaluation can be performed on acquired images, todetermine image statuses of different images, and an image status of apathology image often reflects an operation of a user on a microscopeand a change in a field of view of an image in the microscope, so thatpathology images acquired by using a camera of the microscope may bescreened according to image statuses and a task type, to assist incompleting a task purpose, thereby reducing the difficulty in imageprocessing and improving the efficiency of task processing.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a schematic architectural diagram of an image evaluationsystem according to an embodiment of this application.

FIG. 2 is a schematic flowchart of an image evaluation system accordingto an embodiment of this application.

FIG. 3 is a schematic flowchart of a pathology image-based image statusdetermining method according to an embodiment of this application.

FIG. 4 is a schematic flowchart of motion evaluation on an imageaccording to an embodiment of this application.

FIG. 5 is a schematic coordinate diagram of an image center of a sourceregion image according to an embodiment of this application.

FIG. 6 is a schematic comparison diagram of a source region image and atarget region image according to an embodiment of this application.

FIG. 7 is a schematic flowchart of a method for performing image clarityevaluation on an image according to an embodiment of this application.

FIG. 8 is a schematic flowchart of a pathology image-based processingmethod according to an embodiment of this application.

FIG. 9 is a schematic flowchart of a task of automatically storing animage according to an embodiment of this application.

FIG. 10 is a schematic flowchart of a pathology image-based processingmethod according to an embodiment of this application.

FIG. 11 is a schematic flowchart of a real-time artificial intelligence(AI) auxiliary diagnosis task according to an embodiment of thisapplication.

FIG. 12 is a schematic flowchart of a pathology image-based processingmethod according to an embodiment of this application.

FIG. 13 is a schematic flowchart of a microscope field-of-view remotesharing task according to an embodiment of this application.

FIG. 14 is a schematic diagram of a composition structure of an imagestatus determining apparatus according to an embodiment of thisapplication.

FIG. 15 is a schematic diagram of a composition structure of an imagestatus determining apparatus according to an embodiment of thisapplication.

FIG. 16 is a schematic diagram of a composition structure of an imagestatus determining apparatus according to an embodiment of thisapplication.

FIG. 17 is a schematic diagram of a composition structure of an imagestatus determining apparatus according to an embodiment of thisapplication.

FIG. 18 is a schematic diagram of a composition structure of a terminaldevice according to an embodiment of this application.

DESCRIPTION OF EMBODIMENTS

Embodiments of this application provide a pathology image-based imagestatus determining method and apparatus, and a system, which can performmoving state evaluation and image clarity evaluation on acquired images,so that a reasonable operation may be performed on the images based ondifferent image statuses, thereby reducing the difficulty in imageprocessing and improving the efficiency of task processing.

In the specification of the embodiments of this application, claims, andaccompanying drawings, the terms “first”, “second”, “third”, “fourth”,and the like (if existing) are intended to distinguish between similarobjects, but do not necessarily indicate a specific order or sequence.It is to be understood that data used in this way is interchangeable ina suitable case, so that the embodiments of this application describedherein can be implemented in a sequence in addition to the sequenceshown or described herein. In addition, the terms “include”,“corresponding to” and any other variants are intended to cover thenon-exclusive inclusion. For example, a process, method, system,product, or device that includes a series of steps or units is notnecessarily limited to those expressly listed steps or units, but mayinclude other steps or units not expressly listed or inherent to such aprocess, method, product, or device.

It is to be understood that the pathology image-based image statusdetermining method and the pathology image-based processing methodprovided in this application may be applicable to the field ofartificial intelligence (AI) and may be applicable to the AI-basedmedical field, the field of video monitoring, or the like. For example,in the AI-based medical field, automatic focus is performed by using amicroscope, a current field-of-view remains unchanged, images arecontinuously acquired by using a camera of the microscope, and afocusing button is automatically rotated according to a change of aclear state of the image, to implement automatic focus of themicroscope. In another example, based on the field of video monitoring,a road transportation video may be monitored, and an image in a staticstate is automatically removed, thereby reducing the workload ofsubsequent video analysis.

With the rapid development of science and technologies, the applicationof AI in a medical industry is becoming increasingly extensive, and themost common medical images in the medical field include, but are notlimited to, an angiography image, an angiocardiographic image, acomputerized tomography (CT) image, a B-mode ultrasound image, and apathology image. The pathology image is usually acquired by a smartmicroscope and includes an appearance image and a cell structure imageof a biopsy tissue, and the like.

The main method for performing pathological diagnosis by a doctor is toobserve a slice and observe cell morphology and tissue structure afterenlarging the slice by 40 to 400 times, to make the diagnosis, and asmart microscope and a digital pathology scanner are the most commontools used by the doctor. The smart microscope breaks through alimitation of a conventional microscope and changes from being passivelyused to actively assisting the doctor, for example, assisting the doctorby using a computer vision, from simple but tedious cell metering todifficult and complex cancer type identification and accurate regionaldivision. Meanwhile, smooth human-machine interaction between the doctorand the smart microscope is implemented through voice recognition.Finally, a final pathology report is generated with the assistance of anatural language processing technology. The doctor only needs to give avoice instruction during view of an image, and AI can automatically viewthe image, automatically acquire the image, and assist the doctor indiagnosing. The doctor gives a “generate a report” instruction afterviewing the image, and the smart microscope can fill a report templatewith a microscope screenshot and a diagnosis result and automaticallygenerate a report for the doctor to review the result and issue thereport, so that a report generation step that is the most complexbecomes fast and labor-saving. The smart microscope plays an importantrole in mitotic cell detection, immunohistochemical quantitativeanalysis, cancer region monitoring, and an auxiliary diagnosis process.

For ease of understanding, this application provides a pathologyimage-based image status determining method and a pathology image-basedprocessing method, and both the methods may be applicable to an imageevaluation system shown in FIG. 1. FIG. 1 is a schematic architecturaldiagram of an image evaluation system according to an embodiment of thisapplication, and FIG. 2 is a schematic flowchart of an image evaluationsystem according to an embodiment of this application. Descriptions areprovided below with reference to FIG. 1 and FIG. 2.

Step S1. Acquire a plurality of consecutive (e.g., continuous) imagesusing a camera.

During actual implementation, an image may be acquired using a camera ofa terminal or acquired using a camera that is independent of a terminal.

Step S2. Determine whether a current image is a first image, performstep S3 if the current image is the first image, and otherwise, performstep S4.

During actual implementation, a terminal device determines whether acurrent image is a first image, or a terminal device transmits anacquired image to a server, and the server determines whether thecurrent image is a first image. If the current image is the first image,step S3 is performed, and if the current image is not the first image,step S4 is performed.

Step S3. Determine that the current image is in a moving state.

Step S4. Evaluate the movement state of the current image.

Step S5. Determine, if it is detected that the current image is moving,that the current image is in the moving state.

Step S6. Determine, if it is detected that the current image stopsmoving, that the state of the current image is in a transition statetransitioning from the moving state to a static state.

Step S7. Determine, if it is detected that the current image startsmoving, that the state of the current image is in a transition statetransitioning from the static state to the moving state.

Step S8. Evaluate an image clarity (e.g., whether the image is clear, infocus, out of focus, sharp, crisp, blurred etc.) state of the currentimage if it is detected that the current image is in the static state.

Step S9. Determine, if it is detected that image clarity (e.g., a degreeof clarity, whether the image is clear, in focus, out of focus, sharp,crisp, blurred etc.) of the current image does not change, that thecurrent image is in the static state.

Step S10. Determine, if it is detected that the current image becomesclear, that the state of the current image is in a focusing state and istransitioning from a blurred state to a clear state.

Step S11. Determine, if it is detected that the current image becomesblurred, that the state of the current image is in a focusing state andis transitioning from the clear state to the blurred state.

The terminal device includes, and is not limited to, a smart microscope,a tablet computer, a notebook computer, a palmtop computer, a mobilephone, a speech interaction device, and a personal computer (PC), and isnot limited herein. The smart microscope is integrated with visual,voice, and natural language processing technologies of AI. A doctoreasily enters a voice instruction, and AI can perform automaticrecognition, detection, and quantitative calculation, generate a report,and display a detection result into an eyepiece of the doctor in realtime, to remind the doctor in time without interruption of animage-viewing process, thereby improving the diagnosis efficiency andaccuracy of the doctor.

The pathology image-based image status determining method according tothe embodiments of this application is described below with reference tothe foregoing descriptions. FIG. 3 is a flowchart of a pathologyimage-based image status determining method according to an embodimentof this application. Referring to FIG. 3, the method includes thefollowing steps.

101. Obtain a pathology image set, the pathology image set including atleast a to-be-evaluated image and associated images, the associatedimages and the to-be-evaluated image being consecutive frame images.

In an actual application, the terminal device (for example, a smartmicroscope) acquires a pathology image set using a camera, and an imagestatus determining apparatus obtains the pathology image set. Thepathology image set includes a plurality of consecutive (e.g.,continuous) images, that is, there may be a plurality of frames ofassociated images, and the associated images may be several previousadjacent frames of images that are acquired before the to-be-evaluatedimage.

It may be understood that the image status determining apparatus may bedeployed on the terminal device, for example, deployed on the smartmicroscope, or may be deployed on the server. This is not limitedherein.

102. Determine a first status corresponding to the to-be-evaluated imageaccording to the pathology image set, the first status being used forindicating a motion change of the to-be-evaluated image.

In an actual application, the image status determining apparatusevaluates a moving state of the to-be-evaluated image, to obtain a firststatus, the first status being used for indicating a motion change ofthe to-be-evaluated image. It may be understood that at least threeconsecutive frames of images in the pathology image set are required toevaluate the moving state, that is, the associated images are twoconsecutive frames of images and form the three consecutive frames ofimages with the to-be-evaluated image. For example, the pathology imageset includes the to-be-evaluated image, a previous frame of image of theto-be-evaluated image, and a frame of image prior to the previous frameof image of the to-be-evaluated image.

103. Determine, when the first status is a static state, a second statuscorresponding to the to-be-evaluated image according to the pathologyimage set, the second status being used for indicating a change in imageclarity of the to-be-evaluated image.

In an actual application, if determining that the first status is astatic state, the image status determining apparatus continues toevaluate an image clarity (e.g., a degree of clarity, whether the imageis in focus, out of focus, blurred, sharp, crisp, etc.) of theto-be-evaluated image, to obtain a second status, the second statusindicating a change in image clarity of the to-be-evaluated image. Itmay be understood that at least two consecutive frames of images in thepathology image set are required to evaluate the image clarity state,that is, the associated image is a previous frame of image of theto-be-evaluated image.

For ease of understanding, Table 1 shows the image states (e.g., imagestates, operation states of the microscope) and the descriptionscorresponding to the image statuses based on microscope operations.

TABLE 1 Image states (e.g., operation state) Description Static state Adoctor does not operate a microscope and a field of view of themicroscope is in the static state Moving state The doctor is operatingthe microscope, for example, moving an object stage of the microscope,moving a slide, or switching magnifications, and the field of view ofthe microscope is in the moving state Transition state The doctor stopsoperating the microscope, (Moving state-to-static for example, stopsmoving the object stage state transition) of the microscope, stopsmoving the slide, or completes switching of the magnifications, and thefield of view of the microscope is in transition from the moving stateto the static state. Transition state (Static The doctor startsoperating the microscope, state-to-moving state for example, startsmoving the object stage transition) of the microscope, starts moving theslide, or starts switching the scopes, and the field of view of themicroscope is in transition from the static state to the moving stateFocusing state (from The doctor is adjusting focus, and the fieldblurred to clear) of view of the microscope is gradually becoming clearFocusing state (from The doctor is adjusting the focus, and the clear toblurred) field of view of the microscope is gradually becoming blurred

In some embodiments, the six states described above can be categorizedinto a first status and a second status. The first status includes fourtypes of image states, which are respectively a static state, a movingstate, a first transition state from the moving state to the staticstate, and a second transition state from the static state to the movingstate. The second status includes two types of states, namely a firstfocusing state from clear to blurred and a second focusing state (fromblurred to clear). The six types of image states reflect an operation onthe microscope by an operator (e.g., a doctor) and a change in a fieldof view of the microscope, so that the image is evaluated in real time.

By applying the embodiments of this application, moving state evaluationand image clarity evaluation can be performed on acquired images, todetermine image states (e.g., status) of different images, and an imagestate often reflects a change in a field of view of an image caused byan operation of the user, and plays a very important role in differenttasks, so that pathology images acquired by using a camera of themicroscope may be screened according to the image state and a task type,to assist in completing a task purpose, thereby reducing the difficultyin image processing and improving the efficiency of task processing.

In some embodiments, the associated images include two consecutiveframes of images, and the two consecutive frames of images arerespectively a first associated image and a second associated image.Correspondingly, the first status corresponding to the to-be-evaluatedimage may be determined by the following method:

obtaining a similarity between the to-be-evaluated image and the firstassociated image, the first associated image pertaining to the pathologyimage set, and the first associated image being a previous imageadjacent to the to-be-evaluated image;

obtaining a similarity between the first associated image and the secondassociated image when the similarity between the to-be-evaluated imageand the first associated image is greater than a similarity threshold,the second associated image pertaining to the pathology image set, andthe second associated image is a an image acquired prior to the firstassociated image;

determining, when the similarity between the first associated image andthe second associated image is greater than the similarity threshold,that the first status is a static state; and

determining, when the similarity between the first associated image andthe second associated image is less than or equal to the similaritythreshold, that the first status corresponds to a transition state fromthe moving state to the static state.

In some embodiments, when the associated images include a firstassociated image and a second associated image that are consecutive, thefirst status corresponding to the to-be-evaluated image may bedetermined according to the pathology image set by the following method:

obtaining a similarity between the to-be-evaluated image and the firstassociated image, the first associated image being a previous imageadjacent to the to-be-evaluated image;

obtaining a similarity between the first associated image and the secondassociated image when the similarity between the to-be-evaluated imageand the first associated image is less than or equal to a similaritythreshold, the second associated image being a previous image adjacentto the first associated image;

determining, when the similarity between the first associated image andthe second associated image is greater than the similarity threshold,that the first status (e.g., state) is of static state-to-moving statetransition; and

determining, when the similarity between the first associated image andthe second associated image is less than or equal to the similaritythreshold, that the first status is the moving state.

A method for determining the first status based on similaritycalculation is described below. FIG. 4 is a schematic flowchart ofmotion evaluation on an image according to an embodiment of thisapplication. Referring to FIG. 4, the method includes the followingsteps.

Step A1. Acquire a plurality of consecutive frames of images by using acamera of a microscope.

Step A2. Determine whether a current image (e.g., the to-be-evaluatedimage) moves relative to a previous image (e.g., the first associatedimage), perform step A6 if the current image moves, and perform step A3if the current image does not move.

Herein, the determining method includes: obtaining a similarity betweenthe to-be-evaluated image and the first associated image, anddetermining, when the similarity between the to-be-evaluated image andthe first associated image is greater than a similarity threshold, thatthere is no motion between the to-be-evaluated image and the firstassociated image. In this case, step A3 is performed.

Step A3. Determine whether the previous image (that is, the firstassociated image) moves relative to an image prior to the previous image(that is, the second associated image), perform step A5 if the imageprior to the previous image moves, and perform step A4 if the imageprior to the previous image does not move.

Herein, the determining method includes: obtaining a similarity betweenthe first associated image and the second associated image, anddetermining, when the similarity between the first associated image andthe second associated image is greater than the similarity threshold,that there is no motion between the first associated image and thesecond associated image. In this case, step A4 is performed. When thesimilarity between the first associated image and the second associatedimage is less than or equal to the similarity threshold, it isdetermined that there is a motion between the first associated image andthe second associated image. In this case, step A5 is performed.

Step A4. Determine that the first status of the current image (that is,the to-be-evaluated image) is a static state.

Step A5. Determine that the first status of the current image (that is,the to-be-evaluated image) is in a transition state (e.g., movingstate-to-static state transition).

It may be understood that the similarity threshold may be set to 0.9, ormay be set to another parameter, such as 0.8, 0.75, 0.7, etc., which isnot limited this time.

Step A6. Determine whether the previous image (that is, the firstassociated image) moves relative to the image prior to the previousimage (that is, the second associated image), perform step A8 if theimage prior to the previous image moves, and perform step A7 if theimage prior to the previous image does not move.

Step A7. Determine that the first status of the current image (that is,the to-be-evaluated image) is in static state-to-moving statetransition.

Step A8. Determine that the first status of the current image (that is,the to-be-evaluated image) is the moving state.

In the foregoing manner, an association between two images can beevaluated based on a similarity, to provide a reasonable and reliableimplementation for the solution.

In some embodiments, the similarity between the to-be-evaluated imageand the first associated image may be obtained by the following method:

determining a source region pathology image set according to theto-be-evaluated image, the source region pathology image set including Msource region images, M being an integer greater than 1;

determining a target region pathology image set according to the firstassociated image, the target region pathology image set including Mtarget region images, and sizes of the target region images being lessthan sizes of the source region images;

extracting a first source region image from the source region pathologyimage set, and extracting a first target region image from the targetregion pathology image set;

extracting a second source region image from the source region pathologyimage set and extracting a second target region image from the targetregion pathology image set when both the first source region image andthe first target region image are background images, and determiningwhether the second source region image and the second target regionimage are background images; and

calculating a similarity between the first source region image and thefirst target region image when either the first source region image orthe first target region image is not a background image, and using thecalculated similarity as the similarity between the to-be-evaluatedimage and the first associated image.

Herein, the method for obtaining a similarity between images isdescribed. First, M source region images are selected from theto-be-evaluated image, and the M source region images form a sourceregion pathology image set. FIG. 5 is a schematic coordinate diagram ofan image center of a source region image according to an embodiment ofthis application. Referring to FIG. 5, it is assumed that nine sourceregion images are selected from the to-be-evaluated image, coordinatesof each source region image relative to the entire to-be-evaluated imageare an image center E (0.50, 0.50), an image upper left A (0.25, 0.25),an image lower left G (0.25, 0.75), an image upper right C (0.75, 0.25),an image lower right I (0.75, 0.75), an image upper side B (0.50, 0.25),an image right side F (0 75, 0.50), an image lower side H (0.50, 0.75),and an image left side D (0.25, 0.50) in sequence, a size of each sourceregion image being W*H.

M target region images are selected from the first associated image, andthe M target region images form a target region pathology image set. Itis assumed that nine target region images are selected from the firstassociated image, coordinates of each target region image relative tothe first associated image are an image center E (0.50, 0.50), an imageupper left A (0.25, 0.25), an image lower left G (0.25, 0.75), an imageupper right C (0.75, 0.25), an image lower right I (0.75, 0.75), animage upper side B (0.50, 0.25), an image right side F (0 75, 0.50), animage lower side H (0.50, 0.75), and an image left side D (0.25, 0.50)in sequence, a size of each target region image being w*h and meetingW>w and H>h. For example, W=H=96 and w=h=64 may be set. FIG. 6 is aschematic comparison diagram of a source region image and a targetregion image according to an embodiment of this application. As shown infigure, E is a center of the image, a big rectangle corresponds to asource region image, and a small rectangle corresponds to a targetregion image.

During actual implementation, it is assumed that M is nine, centers ofnine region images need to be traversed. If i=0 is initially set, in ani^(th) cycle a first source region image needs to be extracted from theto-be-evaluated image and a first target region image needs to beextracted from the first associated image according to sizes of theregion images and i^(th) center coordinates, for detecting. A detectionmethod may include performing template matching on the first sourceregion image by using a size of the first target region image as a sizeof a sliding window, and if it is detected that both the first sourceregion image and the first target region image are background images,setting i=i+1 and starting a next round of traversal, to detect whethera second source region image and a second target region image arebackground images. Conversely, if the first source region image is not abackground image, or the first target region image is not a backgroundimage, or both are not background images, a similarity between the tworegion images is calculated by using a template matching method. If thecalculated similarity is greater than a similarity threshold, it isconsidered that there is no motion between the two consecutive frames ofimages, and if the calculated similarity is less than or equal to thesimilarity threshold, it is considered that there is a motion betweenthe two consecutive frames of images, regardless of which case, thetraversal is terminated.

If the M source region images and the M target region images are allbackground images, it is considered that the two frames of images arebackground images, and there is no relative motion therebetween.

In an actual application, a method for calculating a similarity betweenthe first associated image and the second associated image is similar tothe method for calculating the similarity between the to-be-evaluatedimage and the first associated image. Details are not described thistime.

By applying the method for obtaining a similarity between imagesprovided in the embodiments of this application, an image is dividedinto a plurality of regions, and similarity calculation is performed onthe regions instead of on the entire image directly. In this way, on onehand, the accuracy of determining the similarity can be ensured as muchas possible, and if all the regions are background images, the entireimage very probably does not include useful information, and on theother hand, a size of the region is much less than a size of the entireimage, even though the time complexity of the template matching methodis relatively high, the evaluation can also be completed within arelatively short time.

In some embodiments, whether the second source region image and thesecond target region image are background images may be detected (e.g.,determined) by the following method:

calculating a pixel value standard deviation of the second source regionimage;

determining, when the pixel value standard deviation of the secondsource region image is less than or equal to a standard deviationthreshold, that the second source region image is a background image;

calculating a pixel value standard deviation of the second target regionimage; and

determining, when the pixel value standard deviation of the secondtarget region image is less than or equal to the standard deviationthreshold, that the second target region image is a background image.

Herein, the method for determining a background image is described. If asource region image and a target region image are red green blue (RGB)images, the source region image and the target region image need to befirst converted into grayscale images. A pixel value standard deviationof the target region image and a pixel value standard deviation of thesource region image are calculated respectively based on the grayscaleimages. If the pixel value standard deviation is less than or equal to agiven standard deviation threshold, the region image is a backgroundimage. A method for calculating the pixel value standard deviation is asfollows:

${\delta = \sqrt{\frac{1}{M \times N}{\sum\limits_{i = 1}^{M}{\sum\limits_{j = 1}^{N}\left( {{P\left( {i,j} \right)} - \mu} \right)^{2}}}}};$

where δ represents a pixel value standard deviation, M×N represents asize of a region image, P(i, j) represents a pixel value of an i^(th)row and a j^(th) column in the region image, and μ represents an averagevalue.

By applying the method for detecting a background image provided in theembodiments of this application, a change of an image may be betterrepresented by using a pixel value standard deviation, and a dispersiondegree of each pixel in the image is truly reflected, thereby improvingthe detection accuracy.

In some embodiments, the similarity between the first source regionimage and the first target region image may be calculated by thefollowing method:

obtaining an image matrix through calculation according to the firstsource region image and the first target region image, the image matrixincluding a plurality of elements; and

determining the similarity between the first source region image and thefirst target region image according to the image matrix, the similaritybetween the first source region image and the first target region imagebeing a maximum value of the elements in the image matrix.

Herein, the method for calculating a similarity between region images isdescribed, and the first source region image and the first target regionimage are used as examples. In an actual application, the sameprocessing manner may be used for each source region image and eachtarget region image.

During actual implementation, for a source region image and a targetregion image, a size (w*h) of the target region image is less than asize (W*H) of the source region image, and the target region image needsto traverse the entire source region image in a sliding manner. Then,the entire source region image needs to be traversed in the slidingmanner for (W−w+1) times in a horizontal direction and needs to betraversed in the sliding manner for (H−h+1) times in a verticaldirection. Therefore, a result obtained through template matching is animage matrix with a size of (W−w+1)*(H−h+1), which is recorded as R, andthe image matrix may be calculated by the following method:

${{R\left( {x,y} \right)} = \frac{\sum_{x^{\prime},y^{\prime}}{,{{I_{2}^{\prime}\left( {x^{\prime},y^{\prime}} \right)} \cdot {I_{1}^{\prime}\left( {{x + x^{\prime}},{y + y^{\prime}}} \right)}}}}{\sqrt{\sum_{x^{\prime},y^{\prime}}{,{{I_{2}^{\prime}\left( {x^{\prime},y^{\prime}} \right)}^{2} \cdot {\sum_{x^{\prime},y^{\prime}}{I_{1}^{\prime}\left( {{x + x^{\prime}},{y + y^{\prime}}} \right)}^{2}}}}}}},{where}$${{I_{1}^{\prime}\left( {{x + x^{\prime}},{y + y^{\prime}}} \right)} = {{I_{1}\left( {{x + x^{\prime}},{y + y^{\prime}}} \right)} - {\frac{1}{w \cdot h}{\sum_{x^{''},y^{''}}{I_{1}\left( {{x + x^{''}},{y + y^{''}}} \right)}}}}};$${{I_{2}^{\prime}\left( {x^{\prime},y^{\prime}} \right)} = {{I_{2}\left( {x^{\prime},y^{\prime}} \right)} - {\frac{1}{w \cdot h}{\sum_{x^{''},y^{''}}{I_{2}\left( {x^{''},y^{''}} \right)}}}}};$

where R(x, y) represents an element value of a matrix R at (x, y), I₁represents a source region image, I′₁ represents a source region imageafter normalization processing, I₂ represents a target region image, I′₂represents a target region image after the normalization processing, avalue range of x is an integer greater than or equal to 0 and less thanor equal to (W−w), a value range of Y is an integer greater than orequal to 0 and less than or equal to (H−h), a value range of x′ is aninteger greater than or equal to 0 and less than or equal to w, a valuerange y′ is an integer greater than or equal to 0 and less than or equalto h, only a region with a start point of (x, y) and a size of w*h inthe target region image is operated, and the entire source region imageis operated.

Value ranges of the elements in the image matrix are 0 to 1, a maximumvalue is selected as a similarity between two images, and a largersimilarity indicates that the two images are more similar. It may beunderstood that the template matching algorithm adopted in thisapplication is a normalized correlation coefficient matching algorithm(TM_CCOEFF_NORMED). In an actual application, a square differencematching algorithm (CV_TM_SQDIFF), a normalized square differencematching algorithm (CV_TM_SQDIFF_NORMED), a correlation matchingalgorithm (CV_TM_CCORR), a normalized correlation matching algorithm(CV_TM_CCORR NORMED), or a correlation coefficient matching algorithm(CV_TM_CCOEFF) may alternatively be adopted.

The template matching algorithm may effectively distinguish movement ofthe field of view of the microscope from jitter. The tremble of theground or the table causes jitter of the field of view of themicroscope, resulting in a slight offset of two consecutive images, butan offset caused by a man-made motion is usually quite large. Therefore,when the template matching method is used, W*H and w*h need to beproperly set. It may be approximately considered that an offset that isless than (W−w)/2 in the horizontal direction and less than (H−h)/2 inthe vertical direction is a jitter, and an offset that is greater thanor equal to (W−w)/2 in the horizontal direction or greater than or equalto (H−h)/2 in the vertical direction is a motion.

In some embodiments, the second status corresponding to theto-be-evaluated image may be determined according to the pathology imageset by the following method:

obtaining image clarity of the to-be-evaluated image and image clarityof the first associated image, the first associated image pertaining tothe pathology image set, and the first associated image being a previousimage adjacent to the to-be-evaluated image;

obtaining image clarity of a benchmark image when the image clarity ofthe to-be-evaluated image and the image clarity of the first associatedimage meet a first preset condition; and

determining, when the image clarity of the benchmark image and the imageclarity of the to-be-evaluated image meet a second preset condition,that the second status is a focusing state, the focusing state being inclear state-to-blurred state transition or blurred state-to-clear statetransition.

Herein, the method for evaluating image clarity of an image (e.g., animage clarity, a degree of clarity of an image, whether an image is infocus, or blurred, etc.) is described. FIG. 7 is a schematic flowchartof a method for evaluating image clarity of an image according to anembodiment of this application. Referring to FIG. 7, the method includesthe following steps.

Step B1. Acquire a plurality of consecutive frames of images by using acamera of a microscope.

Step B2. Determine whether image clarity of a current image (e.g.,whether the image is clear, in focus, sharp, crisp, blurred, out offocus etc.) (that is, the to-be-evaluated image) is changed relative toimage clarity of a previous image (that is, the first associated image),perform step B3 if the image clarity of the current image is changed,and perform step B4 if the image clarity of the current image is notchanged.

Herein, the determining method includes: first, obtaining image clarity(e.g., whether the image is in focus, sharp, crisp, blurred, etc.) ofthe to-be-evaluated image and image clarity of the first associatedimage, then determining whether the image clarity of the to-be-evaluatedimage and the image clarity of the first associated image meet a firstpreset condition, and determining, when the image clarity of theto-be-evaluated image and the image clarity of the first associatedimage meet the first preset condition, that the image clarity of theto-be-evaluated image and the image clarity of the first associatedimage are changed. In this case, step B3 is performed. Conversely, whenthe image clarity of the to-be-evaluated image and the image clarity ofthe first associated image do not meet the first preset condition, it isdetermined that the image clarity of the to-be-evaluated image and theimage clarity of the first associated image are not changed. In thiscase, step B4 is performed.

Step B3. Determine whether the image clarity (e.g., whether the image isclear, in focus, out of focus, blurred, sharp, crisp, etc.) of thecurrent image (that is, the to-be-evaluated image) is changed relativeto an image clarity of a benchmark image.

Herein, the determining method includes: first, obtaining image clarity(e.g., whether the image is clear, in focus, out of focus, blurred,sharp, crisp, etc.) of the benchmark image, then determining whether theimage clarity of the benchmark image and the image clarity of theto-be-evaluated image meet a second preset condition, and determiningthat the second status is a focusing state when the image clarity of thebenchmark image and the image clarity of the to-be-evaluated image meetthe second preset condition. In this case, if the image becomes blurred,step B5 is performed, and it is determined that the focusing state is inclear state-to-blurred state transition. If the image becomes clear,step B6 is performed, and it is determined that the focusing state is inblurred state-to-clear state transition. If the image clarity of thebenchmark image and the image clarity of the to-be-evaluated image donot meet the second preset condition, step B7 is performed, that is, itis determined that the second status is a static state.

Step B4. Determine that the current image (that is, the to-be-evaluatedimage) is in a static state.

Step B5. Determine that a focusing state is in clear state-to-blurredstate transition.

Step B6. Determine that the focusing state is in blurred state-to-clearstate transition.

Step B7. Determine that the second status is a static state.

Step B8 may be performed based on conditions of step B4, step B5, andstep B6.

Step B8. Update the benchmark image to the current image (that is, theto-be-evaluated image).

By applying the method for evaluating image clarity of an image in realtime provided in the embodiments of this application, a problem that theimage clarity is sensitive to changes in external environments can beresolved by using a benchmark image and dual thresholds, so that whetherfocus of a device is being adjusted can be more reliably inferred.

In some embodiments, after the obtaining image clarity of theto-be-evaluated image and image clarity of the first associated image,the following operations may be further performed:

updating the benchmark image to the to-be-evaluated image when the imageclarity of the to-be-evaluated image and the image clarity of the firstassociated image do not meet the first preset condition; and

updating the image clarity of the benchmark image when the image clarityof the benchmark image and the image clarity of the to-be-evaluatedimage do not meet the second preset condition.

Correspondingly, after determining that the second status is a focusingstate, the following operations may be further performed:

updating the benchmark image to the to-be-evaluated image.

In an actual application, the image clarity is relatively sensitive tothe changes in the external environments, and movement of the device orself-adjustment (for example, automatic exposure or automatic whitebalance) of a camera results in a relatively large change in the imageclarity. In this embodiment of this application, the problem may beresolved by the following method.

During actual implementation, a standard deviation of a Laplacian matrixof an image is used as image clarity. The Laplacian matrix describescontour information of the image. When the field of view of themicroscope remains unchanged, a larger standard deviation of theLaplacian matrix indicates that a contour of the image is clearer andthe image clarity of the image is larger. In addition to using thestandard deviation of the Laplacian matrix as the image clarity, anotherindex such as an average value or an information entropy of theLaplacian matrix may alternatively be adopted.

During image processing, a convolution operation may be performed on animage by using the following 3*3 template, to generate a Laplacianmatrix of the image, and the template is:

$\begin{bmatrix}0 & 1 & 0 \\1 & {- 4} & 1 \\0 & 1 & 0\end{bmatrix}.$

The Laplacian matrix extracts edge information of the image. A clearerimage indicates that an edge of the image is clearer, a value of anelement in the Laplacian matrix fluctuates larger (a value of an elementat a boundary is larger), and a standard deviation is larger.

After the image clarity of the to-be-evaluated image and the imageclarity of the first associated image are obtained, the benchmark imageis updated to the to-be-evaluated image if the image clarity of theto-be-evaluated image and the image clarity of the first associatedimage do not meet the first preset condition. The evaluation isperformed by using the benchmark image and two consecutive images. Whena difference between the image clarity of the to-be-evaluated image andthe image clarity of the benchmark image is less than a given imageclarity resolution threshold (e.g., image clarity threshold, e.g., athreshold value associated with whether the image is in focus, blurred,etc.) (that is, the image clarity of the benchmark image and the imageclarity of the to-be-evaluated image do not meet the second presetcondition), a possible case includes that the doctor does not adjustfocus, the doctor adjusts the focus by an excessively small amplitude,the microscope jitters, the camera is self-adjusting, or the like. Inthis case, instead of updating the benchmark image, the image clarity ofthe benchmark image continues to be accumulated, to make a more accurateinference. An accumulation method is (image clarity)+a or (imageclarity)+b, wherein a and b are positive numbers.

It may be understood that the first preset condition may be that adifference between the image clarity of the to-be-evaluated image andthe image clarity of the first associated image is greater than or equalto the image clarity threshold, and the second preset condition may bethat a difference between the image clarity of the benchmark image andthe image clarity of the to-be-evaluated image is greater than or equalto the image clarity threshold.

By applying the dual-threshold detection manner provided in theembodiments of this application, the evaluation is performed by using abenchmark image and two consecutive images, and when a differencebetween the image clarity of a current image and the image clarity ofthe benchmark image is less than a given threshold, a possible caseincludes that the doctor does not adjust focus, the doctor adjusts thefocus by an excessively small amplitude, the microscope jitters, thecamera is self-adjusting, or the like. In this case, instead of updatingthe benchmark image, an image clarity difference of the benchmark imagecontinues to be accumulated, which helps to obtain a more accuratedetection result.

In some embodiments, after the obtaining image clarity of theto-be-evaluated image and image clarity of the first associated image,the method may further include the following operations:

determining whether a difference between the image clarity of theto-be-evaluated image and the image clarity of the first associatedimage is greater than or equal to a first image clarity threshold;

determining, when the difference between the image clarity of theto-be-evaluated image and the image clarity of the first associatedimage is greater than or equal to the first image clarity threshold,that the image clarity of the to-be-evaluated image and the imageclarity of the first associated image meet the first preset condition;

determining, when the difference between the image clarity of theto-be-evaluated image and the image clarity of the first associatedimage is less than the first image clarity threshold, whether adifference between the image clarity of the benchmark image and theimage clarity of the to-be-evaluated image is greater than or equal to asecond image clarity threshold, the second image clarity threshold beinggreater than the first image clarity threshold; and

determining, when the difference between the image clarity of thebenchmark image and the image clarity of the to-be-evaluated image isgreater than or equal to the second image clarity threshold, that theimage clarity of the benchmark image and the image clarity of theto-be-evaluated image meet the second preset condition.

The embodiments of this application introduce dual thresholds, that is,introduce a first image clarity threshold and a second image claritythreshold. The first image clarity threshold is used when image clarityof a current image and image clarity of a previous image are compared,that is, it is determined whether a difference between the image clarityof the to-be-evaluated image and the image clarity of the firstassociated image is greater than or equal to the first image claritythreshold, and if the difference between the image clarity of theto-be-evaluated image and the image clarity of the first associatedimage is greater than or equal to the first image clarity threshold, itis determined that the image clarity of the to-be-evaluated image andthe image clarity of the first associated image meet the first presetcondition. Conversely, the image clarity of the to-be-evaluated imageand the image clarity of the first associated image do not meet thefirst preset condition.

A high threshold is used when the image clarity of the current image andimage clarity of a benchmark image are compared, that is, it isdetermined whether a difference between the image clarity of theto-be-evaluated image and the image clarity of the benchmark image isgreater than or equal to the second image clarity threshold, and if thedifference between the image clarity of the benchmark image and theimage clarity of the to-be-evaluated image is greater than or equal tothe second image clarity threshold, it is determined that the imageclarity of the benchmark image and the image clarity of theto-be-evaluated image meet the second preset condition. Conversely, theimage clarity of the benchmark image and the image clarity of theto-be-evaluated image do not meet the second preset condition.

Because the jitter changes the blur a lot, when the difference in theimage clarity is greater than the first image clarity threshold, itcannot be inferred whether the doctor is adjusting the focus or themicroscope jitters. Only when the difference in the image clarity isgreater than the second image clarity threshold, it can be inferred thatthe doctor is adjusting the focus. Therefore, it is more reliable to usethe low threshold to infer that the doctor does not adjust the focus ofthe microscope and to use the high threshold to infer that the doctor isadjusting the focus of the microscope.

The first image clarity threshold may be set to 0.02, the first imageclarity threshold is a low threshold, the second image clarity thresholdmay be set to 0.1, and the second image clarity threshold is a highthreshold. In an actual application, the first image clarity thresholdand the second image clarity threshold may be further set to anotherparameter, which are not limited this time.

By applying the dual-threshold detection manner provided in theembodiments of this application, a low threshold is used when imageclarity of a current image and image clarity of a previous image arecompared, and a high threshold is used when the image clarity of thecurrent image and image clarity of a benchmark image are compared. Whenthe low threshold is used, it may be inferred that the doctor does notadjust focus of the microscope, and when the high threshold is used, itis inferred that the doctor is adjusting the focus of the microscope,thereby improving the reliability of image clarity detection.

The pathology image-based processing method according to the embodimentsof this application is described below with reference to the foregoingdescriptions. During actual implementation, the method may beindependently implemented by a terminal (for example, a smartmicroscope) or a server, or may be implemented by a terminal and aserver in cooperation, and an example in which the method isindependently implemented by the terminal is used. FIG. 8 is a schematicflowchart of a pathology image-based processing method according to anembodiment of this application. Referring to FIG. 8, the method includesthe following steps.

201. A terminal obtains a pathology image set, the pathology image setincluding a to-be-evaluated image, a first associated image, and asecond associated image, the first associated image being a previousimage adjacent to the to-be-evaluated image, and the second associatedimage being a previous image adjacent to the first associated image.

In an actual application, a smart microscope acquires a pathology imageset by using a camera, to obtain the pathology image set. The pathologyimage set includes a plurality of consecutive pathology images, that is,includes at least one to-be-evaluated image and a plurality ofassociated images, and the associated images refer to several previousadjacent frames of images before the to-be-evaluated image.

It may be understood that the smart microscope may alternativelytransmit the pathology image set to the server, and the serverdetermines an image status corresponding to the to-be-evaluated image.

202. Determine a first status corresponding to the to-be-evaluated imageaccording to the pathology image set, the first status being used forindicating a motion change of the to-be-evaluated image.

In an actual application, the smart microscope or the server evaluates amoving state of the to-be-evaluated image, to obtain a first status, thefirst status being used for indicating a motion change of theto-be-evaluated image. It may be understood that at least three framesof pathology images in the pathology image set are required to evaluatethe moving state, that is, the to-be-evaluated image, a previous frameof pathology image of the to-be-evaluated image (that is, the firstassociated image), and a frame of pathology image prior to the previousframe of pathology frame of the to-be-evaluated image (that is, thesecond associated image) are included.

203. Store the to-be-evaluated image when the first status is of movingstate-to-static state transition.

In an actual application, the to-be-evaluated image is stored when it isdetermined that the first status of the to-be-evaluated image is inmoving state-to-static state transition.

204. Determine, when the first status is a static state, a second statuscorresponding to the to-be-evaluated image according to the pathologyimage set, the second status being used for indicating a change in imageclarity of the to-be-evaluated image.

In an actual application, if it is determined that the first status ofthe to-be-evaluated image is the static state, an image clarity (e.g.,whether the image is in focus, blurry, sharp, etc.) an image clarity(e.g., whether the image is in focus, blurry, sharp, etc.) of theto-be-evaluated image continues to be evaluated, to obtain a secondstatus, the second status indicating a change in image clarity of theto-be-evaluated image. It may be understood that at least two frames ofpathology images in the pathology image set are required to evaluate theimage clarity state, that is, the to-be-evaluated image and a previousframe of pathology image of the to-be-evaluated image (that is, thefirst associated image) are included.

205. Store the to-be-evaluated image when the second status is ofblurred state-to-clear state transition.

In an actual application, the to-be-evaluated image is stored when it isdetermined that the second status of the to-be-evaluated image is inblurred state-to-clear state transition.

FIG. 9 is a schematic flowchart of a method for a task of automaticallystoring an image according to an embodiment of this application.Referring to FIG. 9, a large quantity of pathology images are firstacquired by using a camera of a smart microscope, then image statuses ofthe pathology images are evaluated, that is, a moving state and an imageclarity (e.g., whether the image is in focus, blurry, sharp, etc.) animage clarity (e.g., whether the image is in focus, blurry, sharp, etc.)are evaluated, and pathology images in six types of image statuses maybe obtained based on an evaluation result. The six types of imagestatuses include: a static state, a moving state, moving state-to-staticstate transition, static state-to-moving state transition, a focusing(clear state-to-blurred state transition) state, and a focusing (blurredstate-to-clear state transition) state. In an actual application, only apathology image in moving state-to-static state transition and apathology image in the focusing (blurred state-to-clear statetransition) state are stored, by which the pathology images areautomatically stored.

With the application of the foregoing embodiments of this application,the pathology image acquired by the smart microscope may beautomatically stored, and the pathology image is used for subsequentpathology report, communication, backup, and the like. Based on the taskof automatically storing an image, a pathology image set is screened,and because an image in another state is redundant or of a low quality,only an image in moving state-to-static state transition and an image inblurred state-to-clear state transition need to be stored. In theforegoing manner, on one hand, there is no need for medical staff tomanually acquire images, which improves the work efficiency, and on theother hand, a storage space occupied by the images is reduced.

The pathology image-based processing method according to the embodimentsof this application is described below with reference to the foregoingdescriptions. FIG. 10 is a schematic flowchart of a pathologyimage-based processing method according to an embodiment of thisapplication. Referring to FIG. 10, the method includes the followingsteps.

301. Obtain a pathology image set, the pathology image set including ato-be-evaluated image, a first associated image, and a second associatedimage, the first associated image being a previous image adjacent to theto-be-evaluated image, and the second associated image being a previousimage adjacent to the first associated image.

In an actual application, a smart microscope acquires a pathology imageset by using a camera, to obtain the pathology image set. The pathologyimage set includes a plurality of consecutive pathology images, that is,includes at least one to-be-evaluated image and a plurality ofassociated images, and the associated images refer to several previousadjacent frames of images before the to-be-evaluated image. That is, thefirst associated image is a previous image adjacent to theto-be-evaluated image, and the second associated image is a previousimage adjacent to the first associated image.

It may be understood that the smart microscope may alternativelytransmit the pathology image set to the server, and the serverdetermines an image status corresponding to the to-be-evaluated image.

302. Determine a first status corresponding to the to-be-evaluated imageaccording to the pathology image set, the first status being used forindicating a motion change of the to-be-evaluated image.

In an actual application, the smart microscope or the server evaluates amoving state of the to-be-evaluated image, to obtain a first status, thefirst status being used for indicating a motion change of theto-be-evaluated image. It may be understood that at least three framesof pathology images in the pathology image set are required to evaluatethe moving state, that is, the to-be-evaluated image, a previous frameof pathology image of the to-be-evaluated image (that is, the firstassociated image), and a frame of pathology image prior to the previousframe of pathology frame of the to-be-evaluated image (that is, thesecond associated image) are included.

303. Perform artificial intelligence (AI) diagnosis on theto-be-evaluated image when the first status is of moving state-to-staticstate transition.

In an actual application, AI auxiliary diagnosis is performed on theto-be-evaluated image when it is determined that the first status of theto-be-evaluated image is in moving state-to-static state transition.

A clinical decision support system is a support system configured toassist the doctor in making decision during diagnosis. The systemanalyzes data of a patient, to provide a diagnosis suggestion for thedoctor, and then the doctor performs determining by combining theprofession of the doctor, so that the diagnosis is faster and moreaccurate. The application of AI in the field of diagnosis is mainlyaimed at problems that a growth rate of radiologists is lower than agrowth speed of image data, allocation of medical talent resources isuneven, and a misdiagnosis rate is relatively high. AI may be used foranalyzing case data and providing a diagnosis suggestion for the patientmore reliably, thereby saving time for the doctor.

304. Determine, when the first status is a static state, a second statuscorresponding to the to-be-evaluated image according to the pathologyimage set, the second status being used for indicating a change in imageclarity of the to-be-evaluated image.

In an actual application, if it is determined that the first status ofthe to-be-evaluated image is the static state, an image clarity (e.g.,whether the image is in focus, blurry, sharp, etc.) an image clarity(e.g., whether the image is in focus, blurry, sharp, etc.) of theto-be-evaluated image continues to be evaluated, to obtain a secondstatus, the second status indicating a change in image clarity of theto-be-evaluated image. It may be understood that at least two frames ofpathology images in the pathology image set are required to evaluate theimage clarity state, that is, the to-be-evaluated image and a previousframe of pathology image of the to-be-evaluated image (that is, thefirst associated image) are included.

305. Perform AI diagnosis on the to-be-evaluated image when the secondstatus is of blurred state-to-clear state transition.

In an actual application, AI auxiliary diagnosis is performed on theto-be-evaluated image when it is determined that the second status ofthe to-be-evaluated image is in blurred state-to-clear state transition.

For ease of description, FIG. 11 is a schematic flowchart of a real-timeAI auxiliary diagnosis task according to an embodiment of thisapplication. As shown in the figure, a large quantity of pathologyimages are first acquired by using a camera of a smart microscope, thenimage statuses of the pathology images are evaluated, that is, a movingstate and an image clarity (e.g., whether the image is in focus, blurry,sharp, etc.) are evaluated, and pathology images in six types of imagestatuses may be obtained based on an evaluation result. The six types ofimage states include: a static state, a moving state, a first transitionstate (e.g., transition from moving state to static state), a secondtransition state (e.g., transition from static state to moving state), afirst focusing state (e.g., from clear to blurred), and a secondfocusing state (e.g., from blurred to clear). In an actual application,AI auxiliary diagnosis is performed on only a pathology image in atransition state (e.g., transition from moving state to static state)and a pathology image in a focusing state (e.g., from blurred to clear).

Herein, real-time AI auxiliary diagnosis based on an image refers totransmitting images acquired by using the camera to an AI auxiliarydiagnosis module in real time when the doctor uses a pathologicalmicroscope, and feeding back an AI auxiliary diagnosis result to thedoctor, thereby improving the work efficiency of the doctor. In thisembodiment of this application, the images transmitted to the AIauxiliary diagnosis module may be screened, and only an image in movingstate-to-static state transition and an image in blurred state-to-clearstate transition are selected. This is because the images in the twostates are what the doctor needs to observe carefully and is reallyinterested in, thereby greatly reducing a throughput pressure of the AIauxiliary diagnosis module.

The pathology image-based processing method according to the embodimentsof this application continues to be described. FIG. 12 is a schematicflowchart of a pathology image-based processing method according to anembodiment of this application. Referring to FIG. 12, the methodincludes the following steps.

401. Obtain a pathology image set, the pathology image set including ato-be-evaluated image, a first associated image, and a second associatedimage, the first associated image being a previous image adjacent to theto-be-evaluated image, and the second associated image being a previousimage adjacent to the first associated image.

In an actual application, a smart microscope acquires a pathology imageset by using a camera, to obtain the pathology image set. The pathologyimage set includes a plurality of consecutive pathology images, that is,includes at least one to-be-evaluated image and a plurality ofassociated images, and the associated images refer to several previousadjacent frames of images before the to-be-evaluated image. That is, thefirst associated image is a previous image adjacent to theto-be-evaluated image, and the second associated image is a previousimage adjacent to the first associated image.

It may be understood that the smart microscope may alternativelytransmit the pathology image set to the server, and the serverdetermines an image status corresponding to the to-be-evaluated image.

402. Determine a first status corresponding to the to-be-evaluated imageaccording to the pathology image set, the first status being used forindicating a motion change of the to-be-evaluated image.

In an actual application, the smart microscope or the server evaluates amoving state of the to-be-evaluated image, to obtain a first status, thefirst status being used for indicating a motion change of theto-be-evaluated image. It may be understood that at least three framesof pathology images in the pathology image set are required to evaluatethe moving state, that is, the to-be-evaluated image, a previous frameof pathology image of the to-be-evaluated image (that is, the firstassociated image), and a frame of pathology image prior to the previousframe of pathology image of the to-be-evaluated image (that is, thesecond associated image) are included.

403. Transmit the to-be-evaluated image when the first status is ofmoving state-to-static state transition, or the first status is ofstatic state-to-moving state transition, or the first status is a movingstate.

In an actual application, the to-be-evaluated image is transmitted whenit is determined that the first status of the to-be-evaluated image is anon-static state (for example, in moving state-to-static statetransition, in static state-to-moving state transition, or a movingstate).

404. Determine, when the first status is a static state, a second statuscorresponding to the to-be-evaluated image according to the pathologyimage set, the second status being used for indicating a change in imageclarity of the to-be-evaluated image.

In an actual application, if it is determined that the first status ofthe to-be-evaluated image is the static state, an image clarity (e.g.,whether the image is in focus, blurry, sharp, etc.) of theto-be-evaluated image continues to be evaluated, to obtain a secondstatus, the second status indicating a change in image clarity of theto-be-evaluated image. It may be understood that at least two frames ofpathology images in the pathology image set are required to evaluate theimage clarity state, that is, the to-be-evaluated image and a previousframe of pathology image of the to-be-evaluated image (that is, thefirst associated image) are included.

405. Transmit the to-be-evaluated image when the second status is ofblurred state-to-clear state transition or the second status is of clearstate-to-blurred state transition.

In an actual application, the to-be-evaluated image is transmitted whenit is determined that the second status of the to-be-evaluated image isin blurred state-to-clear state transition or clear state-to-blurredstate transition.

For ease of description, FIG. 13 is a schematic flowchart of amicroscope field-of-view remote sharing task according to an embodimentof this application. As shown in the figure, a large quantity ofpathology images are first acquired by using a camera of a smartmicroscope, then image statuses of the pathology images are evaluated,that is, a moving state and an image clarity (e.g., whether the image isin focus, blurry, sharp, etc.) are evaluated, and pathology images insix types of image statuses may be obtained based on an evaluationresult. The six types of image statuses include: a static state, amoving state, moving state-to-static state transition, staticstate-to-moving state transition, a focusing (clear state-to-blurredstate transition) state, and a focusing (blurred state-to-clear statetransition) state. In an actual application, any pathology image in anon-static state may be transmitted.

In an actual application, during hospital consultation or communication,a doctor operating a microscope needs to remotely share a field of viewof the microscope to other doctors for observing. In this case,pathology images continuously acquired by using a camera of themicroscope need to be transmitted to the other party in real time over anetwork. In this way, the pathology images may be screened before beingtransmitted over the network, and a pathology image in a static state isexcluded because the pathology image of this state is redundant, therebyreducing the amount of data required to be transmitted over the network.

The following describes an image status determining apparatus providedin the embodiments of this application in detail. FIG. 14 is a schematicdiagram of a composition structure of an image status determiningapparatus according to an embodiment of this application, and an imagestatus determining apparatus 50 includes:

an obtaining module 501, configured to obtain a pathology image set, thepathology image set including at least a to-be-evaluated image andassociated images, the associated images and the to-be-evaluated imagebeing consecutive frame images;

a determining module 502, configured to determine a first statuscorresponding to the to-be-evaluated image according to the pathologyimage set obtained by the obtaining module 501, the first status beingused for indicating a motion change of the to-be-evaluated image; and

the determining module 502, further configured to determine, when thefirst status is a static state, a second status corresponding to theto-be-evaluated image according to the pathology image set, the secondstatus being used for indicating a change in image clarity of theto-be-evaluated image.

In the foregoing manner, moving state evaluation and image clarity stateevaluation can be performed on acquired images, to determine imagestatuses of different images, and an image status often reflects achange in a field of view of the image caused by an operation of a user,and plays a very important role in different tasks, so that a reasonableoperation may be performed on the images based on different imagestatuses, thereby reducing the difficulty in image processing andimproving the efficiency of task processing.

In some embodiments, the associated images include two consecutiveframes of images, and the two consecutive frames of images arerespectively a first associated image and a second associated image.

The determining module 502 is further configured to obtain a similaritybetween the to-be-evaluated image and the first associated image, thefirst associated image being a previous image adjacent to theto-be-evaluated image;

obtain a similarity between the first associated image and the secondassociated image when the similarity between the to-be-evaluated imageand the first associated image is greater than a similarity threshold,the second associated image pertaining to the pathology image set, andthe second associated image being a previous image adjacent to the firstassociated image;

determine, when the similarity between the first associated image andthe second associated image is greater than the similarity threshold,that the first status is the static state; and

determine, when the similarity between the first associated image andthe second associated image is less than or equal to the similaritythreshold, that the first status is of moving state-to-static statetransition.

In the foregoing manner, an association between two images can beevaluated based on a similarity, to provide a reasonable and reliableimplementation for the solution.

In some embodiments, the associated images include two consecutiveframes of images, and the two consecutive frames of images arerespectively a first associated image and a second associated image.

The determining module 502 is further configured to obtain a similaritybetween the to-be-evaluated image and the first associated image, thefirst associated image being a previous image adjacent to theto-be-evaluated image;

obtain a similarity between the first associated image and the secondassociated image when the similarity between the to-be-evaluated imageand the first associated image is less than or equal to the similaritythreshold, the second associated image pertaining to the pathology imageset, and the second associated image being a previous image adjacent tothe first associated image;

determine, when the similarity between the first associated image andthe second associated image is greater than the similarity threshold,that the first status is of static state-to-moving state transition; and

determine, when the similarity between the first associated image andthe second associated image is less than or equal to the similaritythreshold, that the first status is the moving state.

In the foregoing manner, an association between two images can beevaluated based on a similarity, to provide a reasonable and reliableimplementation for the solution.

In some embodiments, the determining module 502 is further configured todetermine a source region pathology image set according to theto-be-evaluated image, the source region pathology image set including Msource region images, M being an integer greater than 1;

determine a target region pathology image set according to the firstassociated image, the target region pathology image set including Mtarget region images, and sizes of the target region images being lessthan sizes of the source region images;

extract a first source region image from the source region pathologyimage set, and extract a first target region image from the targetregion pathology image set;

extract a second source region image from the source region pathologyimage set and extract a second target region image from the targetregion pathology image set when both the first source region image andthe first target region image are background images, and detect whetherthe second source region image and the second target region image arebackground images; and

calculate a similarity between the first source region image and thefirst target region image when either the first source region image orthe first target region image is not a background image.

By applying the method for obtaining a similarity between imagesprovided in the embodiments of this application, an image is dividedinto a plurality of regions, and similarity calculation is performed onthe regions instead of on the entire image directly. In this way, on onehand, the accuracy of determining the similarity can be ensured as muchas possible, and if all the regions are background images, the entireimage very probably does not include useful information, and on theother hand, a size of the region is much less than a size of the entireimage, even though the time complexity of the template matching methodis relatively high, the evaluation can also be completed within arelatively short time.

In some embodiments, the determining module 502 is further configured tocalculate a pixel value standard deviation of the second source regionimage;

determine, when the pixel value standard deviation of the second sourceregion image is less than or equal to a standard deviation threshold,that the second source region image is a background image.

calculate a pixel value standard deviation of the second target regionimage; and

determine, when the pixel value standard deviation of the second targetregion image is less than or equal to the standard deviation threshold,that the second target region image is a background image.

In this embodiment of this application, a change of an image may bebetter represented by using a pixel value standard deviation, and adispersion degree of each pixel in the image is truly reflected, therebyimproving the detection accuracy.

In some embodiments, the determining module 502 is further configured toobtain an image matrix through calculation according to the first sourceregion image and the first target region image, the image matrixincluding a plurality of elements; and

determine the similarity between the first source region image and thefirst target region image according to the image matrix, the similaritybetween the first source region image and the first target region imagebeing a maximum value of the elements in the image matrix.

By applying the method for calculating a similarity between regionimages provided in the embodiments of this application, a specificoperation manner is provided for implementation of the solution, therebyimproving the feasibility and operability of the solution.

In some embodiments, the determining module 502 is further configured toobtain image clarity of the to-be-evaluated image and image clarity ofthe first associated image, the first associated image pertaining to thepathology image set, and the first associated image being a previousimage adjacent to the to-be-evaluated image;

obtain image clarity of a benchmark image when the image clarity of theto-be-evaluated image and the image clarity of the first associatedimage meet a first preset condition; and

determine that the second status is a focusing state when the imageclarity of the benchmark image and the image clarity of theto-be-evaluated image meet a second preset condition, the focusing statebeing in clear state-to-blurred state transition or blurredstate-to-clear state transition.

In this embodiment of this application, a problem that the image imageclarity is sensitive to changes in external environments can be resolvedby using a benchmark image and dual thresholds, so that whether focus ofa device is being adjusted can be more reliably inferred.

In some embodiments, the determining module 502 is further configured toafter the image clarity of the to-be-evaluated image and the imageclarity of the first associated image are obtained, update the benchmarkimage to the to-be-evaluated image when the image clarity of theto-be-evaluated image and the image clarity of the first associatedimage do not meet the first preset condition;

update the image clarity of the benchmark image when the image clarityof the benchmark image and the image clarity of the to-be-evaluatedimage do not meet the second preset condition; and

update the benchmark image to the to-be-evaluated image after it isdetermined that the second status is the focusing state.

In this embodiment of this application, the evaluation is performed byusing a benchmark image and two consecutive images, and when adifference between image clarity of a current image and image clarity ofthe benchmark image is less than a given threshold, a possible caseincludes that the doctor does not adjust focus, the doctor adjusts thefocus by an excessively small amplitude, the microscope jitters, thecamera is self-adjusting, or the like. In this case, instead of updatingthe benchmark image, a image clarity difference of the benchmark imagecontinues to be accumulated, which helps to obtain a more accuratedetection result.

In some embodiments, the determining module 502 is further configured toafter the image clarity of the to-be-evaluated image and the imageclarity of the first associated image are obtained, determine whether adifference between the image clarity of the to-be-evaluated image andthe image clarity of the first associated image is greater than or equalto a first image clarity threshold;

determine, when the difference between the image clarity of theto-be-evaluated image and the image clarity of the first associatedimage is greater than or equal to the first image clarity threshold,that the image clarity of the to-be-evaluated image and the imageclarity of the first associated image meet the first preset condition;

determine whether a difference between the image clarity of thebenchmark image and the image clarity of the to-be-evaluated image isgreater than or equal to a second image clarity threshold when thedifference between the image clarity of the to-be-evaluated image andthe image clarity of the first associated image is less than the firstimage clarity threshold, the second image clarity threshold beinggreater than the first image clarity threshold; and

determine, when the difference between the image clarity of thebenchmark image and the image clarity of the to-be-evaluated image isgreater than or equal to the second image clarity threshold, that theimage clarity of the benchmark image and the image clarity of theto-be-evaluated image meet the second preset condition.

In this embodiment of this application, a low threshold is used whenimage clarity of a current image and image clarity of a previous imageare compared, and a high threshold is used when the image clarity of thecurrent image and image clarity of a benchmark image are compared. Whenthe low threshold is used, it may be inferred that the doctor does notadjust focus of the microscope, and when the high threshold is used, itis inferred that the doctor is adjusting the focus of the microscope,thereby improving the reliability of image clarity detection.

In some embodiments, FIG. 15 is a schematic diagram of a compositionstructure of an image status determining apparatus 50 according to anembodiment of this application. Referring to FIG. 15, based on FIG. 14,the image status determining apparatus 50 further includes a storagemodule 503.

The storage module 503 is configured to after the determining module 502determines a second status corresponding to the to-be-evaluated imageaccording to the pathology image set, store the to-be-evaluated imagewhen the first status is of moving state-to-static state transition; and

store the to-be-evaluated image when the second status is of blurredstate-to-clear state transition.

In this embodiment of this application, a pathology image acquired by asmart microscope may be automatically stored, and the pathology image isused for subsequent pathology report, communication, backup, and thelike. Based on the task of automatically storing an image, a pathologyimage set is screened, and because an image in another state isredundant or of a low quality, only an image in moving state-to-staticstate transition and an image in blurred state-to-clear state transitionneed to be stored. In the foregoing manner, on one hand, there is noneed for medical staff to manually acquire images, which improves thework efficiency, and on the other hand, a storage space occupied by theimages is reduced.

In some embodiments, FIG. 16 is a schematic diagram of a compositionstructure of an image status determining apparatus 50 according to anembodiment of this application. Referring to FIG. 16, based on FIG. 14,the image status determining apparatus 50 further includes a diagnosismodule 504.

The diagnosis module 504 is configured to after the determining module502 determines a second status corresponding to the to-be-evaluatedimage according to the pathology image set, perform pathologicalanalysis on the to-be-evaluated image when the first status is of movingstate-to-static state transition; and

perform pathological analysis on the to-be-evaluated image when thesecond status is of blurred state-to-clear state transition.

In this embodiment of this application, real-time AI auxiliary diagnosisbased on an image refers to transmitting images acquired by using acamera to an AI auxiliary diagnosis module in real time when the doctoruses a pathological microscope, and feeding back an AI auxiliarydiagnosis result to the doctor, thereby improving the work efficiency ofthe doctor. In this application, the images transmitted to the A1auxiliary diagnosis module may be screened, and only an image in movingstate-to-static state transition and an image in blurred state-to-clearstate transition are selected. This is because the images in the twostates are what the doctor needs to observe carefully and is reallyinterested in, thereby greatly reducing a throughput pressure of the AIauxiliary diagnosis module.

In some embodiments, FIG. 17 is a schematic diagram of a compositionstructure of an image status determining apparatus 50 according to anembodiment of this application. Referring to FIG. 17, based on FIG. 14,the image status determining apparatus 50 further includes atransmission module 505.

The transmission module 505 is configured to after the determiningmodule 502 determines a second status corresponding to theto-be-evaluated image according to the pathology image set, transmit theto-be-evaluated image when the first status is of moving state-to-staticstate transition, or the first status is of static state-to-moving statetransition, or the first status is the moving state; and

transmit the to-be-evaluated image when the second status is of blurredstate-to-clear state transition or the second status is of clearstate-to-blurred state transition.

In this embodiment of this application, during hospital consultation orcommunication, a doctor operating a microscope needs to remotely share afield of view of the microscope to other doctors for observing. In thiscase, pathology images continuously acquired by using a camera of themicroscope need to be transmitted to the other party in real time over anetwork. In this way, the pathology images may be screened before beingtransmitted over the network, and a pathology image in a static state isexcluded because the pathology image of this state is redundant, therebyreducing the amount of data required to be transmitted over the network.

During actual implementation, the image status determining apparatusprovided in the embodiments of this application may be implemented by aterminal device. FIG. 18 is a schematic diagram of a compositionstructure of a terminal device according to an embodiment of thisapplication. As shown in FIG. 18, for ease of description, only partsrelated to the embodiments of this application are shown. For specifictechnical details that are not disclosed, refer to the method part ofthe embodiments of this application. The terminal device may be anyterminal device including a mobile phone, a tablet computer, a personaldigital assistant (PDA), a point of sales (POS), an on-board computer,or the like, and the terminal device being a mobile phone is used as anexample.

FIG. 18 is a block diagram of the structure of a part of a mobile phonerelated to a terminal device according to an embodiment of thisapplication. Referring to FIG. 18, the mobile phone includes componentssuch as: a radio frequency (RF) circuit 910, a memory 920, an input unit930, a display unit 940, a sensor 950, an audio circuit 960, a wirelessfidelity (Wi-Fi) module 970, a processor 980, and a power supply 990. Aperson skilled in the art may understand that the structure of themobile phone shown in FIG. 18 does not constitute any limitation on themobile phone, and instead, the mobile phone may include components moreor fewer than those shown in the figure, or combine some components, orhave a different component arrangement.

The following makes a specific description of the components of themobile phone with reference to FIG. 18.

The RF circuit 910 may be configured to receive and transmit signals inan information receiving and transmitting process or a call process.Specifically, the RF circuit receives downlink information from a basestation, then delivers the downlink information to the processor 980 forprocessing, and transmits designed uplink data to the base station.Generally, the RF circuit 910 includes, but is not limited to, anantenna, at least one amplifier, a transceiver, a coupler, a low noiseamplifier (LNA), and a duplexer. In addition, the RF circuit 910 mayalso communicate with a network and another device through wirelesscommunication. The wireless communication may use any communicationstandard or protocol, including but not limited to global system formobile communication (GSM), general packet radio service (GPRS), codedivision multiple access (CDMA), wideband code division multiple access(WCDMA), long term evolution (LTE), email, short messaging service(SMS), and the like.

The memory 920 may be configured to store a software program and module.The processor 980 runs the software program and module stored in thememory 920, to implement various functional applications and dataprocessing of the mobile phone. The memory 920 may mainly include aprogram storage area and a data storage area, where the program storagearea may store an operating system, an application program required byat least one function (for example, a sound playback function and animage display function), and the like; and the data storage area maystore data (for example, audio data and a phone book) created accordingto use of the mobile phone, and the like. In addition, the memory 920may include a high-speed random access memory, and may further include anonvolatile memory, such as at least one magnetic disk storage device, aflash memory device, or another volatile solid-state storage device.

In some embodiments, the processor 920 is further configured to store acomputer program, the computer program being configured to perform thepathology image-based image status determining method provided in theembodiments of this application.

The input unit 930 may be configured to receive input digit or characterinformation, and generate a key signal input related to the user settingand function control of the mobile phone. During actual implementation,the input unit 930 may include a touch panel 931 and another inputdevice 932. The touch panel 931 may also be referred to as a touchscreen, and may collect a touch operation of a user on or near the touchpanel (such as an operation of a user on or near the touch panel 931 byusing any suitable object or attachment, such as a finger or a touchpen), and drive a corresponding connection apparatus according to apreset program. In some embodiments, the touch panel 931 may include twoparts: a touch detection apparatus and a touch controller. The touchdetection apparatus detects a touch position of the user, detects asignal generated by the touch operation, and transfers the signal to thetouch controller. The touch controller receives touch information fromthe touch detection apparatus, converts the touch information into touchpoint coordinates, and transmits the touch point coordinates to theprocessor 980. Moreover, the touch controller can receive and execute acommand transmitted from the processor 980. In addition, the touch panel931 may be implemented by using various types, such as a resistive type,a capacitive type, an infrared type, and a surface acoustic wave type.In addition to the touch panel 931, the input unit 930 may furtherinclude the another input device 932. During actual implementation, theanother input device 932 may include, but not limited to, one or more ofa physical keyboard, a functional key (for example, a volume control keyor a switch key), a track ball, a mouse, and a joystick.

The display unit 940 may be configured to display information input bythe user or information provided for the user, and various menus of themobile phone. The display unit 940 may include a display panel 941. Insome embodiments, the display panel 941 may be configured in the form ofa liquid crystal display (LCD), an organic light-emitting diode (OLED),or the like. Further, the touch panel 931 may cover the display panel941. After detecting a touch operation on or near the touch panel, thetouch panel 931 transfers the touch operation to the processor 980, todetermine a type of a touch event. Then, the processor 980 provides acorresponding visual output on the display panel 941 according to thetype of the touch event. Although in FIG. 18, the touch panel 931 andthe display panel 941 are used as two separate parts to implement inputand output functions of the mobile phone, in some embodiments, the touchpanel 931 and the display panel 941 may be integrated to implement theinput and output functions of the mobile phone.

The mobile phone may further include at least one sensor 950 such as anoptical sensor, a motion sensor, and other sensors. During actualimplementation, the optical sensor may include an ambient light sensorand a proximity sensor. The ambient light sensor may adjust luminance ofthe display panel 941 according to brightness of the ambient light. Theproximity sensor may switch off the display panel 941 and/or backlightwhen the mobile phone is moved to the ear. As one type of motion sensor,an acceleration sensor can detect magnitude of accelerations in variousdirections (generally on three axes), may detect magnitude and adirection of the gravity when static, and may be applied to anapplication that recognizes the attitude of the mobile phone (forexample, switching between landscape orientation and portraitorientation, a related game, and magnetometer attitude calibration), afunction related to vibration recognition (such as a pedometer and aknock), and the like. Other sensors, such as a gyroscope, a barometer, ahygrometer, a thermometer, and an infrared sensor, may be configured inthe mobile phone. Details are not described herein again.

The audio circuit 960, a speaker 961, and a microphone 962 may provideaudio interfaces between the user and the mobile phone. The audiocircuit 960 may convert received audio data into an electrical signaland transmit the electrical signal to the speaker 961. The speaker 961converts the electrical signal into a sound signal for output. On theother hand, the microphone 962 converts a collected sound signal into anelectrical signal. The audio circuit 960 receives the electrical signal,converts the electrical signal into audio data, and outputs the audiodata to the processor 980 for processing. Then, the processor transmitsthe audio data to, for example, another mobile phone by using the RFcircuit 910, or outputs the audio data to the memory 920 for furtherprocessing.

Wi-Fi is a short distance wireless transmission technology. The mobilephone may help, by using the Wi-Fi module 970, a user to receive andtransmit an email, browse a web page, access stream media, and the like.This provides wireless broadband Internet access for the user. AlthoughFIG. 18 shows the Wi-Fi module 970, it may be understood that the Wi-Fimodule is not a necessary component of the mobile phone, and the Wi-Fimodule may be omitted as required provided that the scope of the essenceof the present disclosure is not changed.

The processor 980 is the control center of the mobile phone, and isconnected to various parts of the mobile phone by using variousinterfaces and lines. By running or executing the software programand/or module stored in the memory 920, and invoking data stored in thememory 920, the processor performs various functions and data processingof the mobile phone, thereby performing overall monitoring on the mobilephone. In some embodiments, the processor 980 may include one or moreprocessing units. In some embodiments, the processor 980 may integratean application processor and a modem processor. The applicationprocessor mainly processes an operating system, a user interface, anapplication program, and the like. The modem processor mainly processeswireless communication. It may be understood that the foregoing modemprocessor may either not be integrated into the processor 980.

The mobile phone further includes the power supply 990 (such as abattery) for supplying power to the components. In some embodiments, thepower supply may be logically connected to the processor 980 by using apower management system, thereby implementing functions such ascharging, discharging and power consumption management by using thepower management system. Although not shown in the figure, the mobilephone may further include a camera, a Bluetooth module, and the like,which are not further described herein.

In this embodiment of this application, the processor 980 included inthe terminal device is further configured to execute a computer programstored in the memory 920, to implement the pathology image-based imagestatus determining method provided in the embodiments of thisapplication.

A person skilled in the art can clearly understand that for convenienceand conciseness of description, for specific working processes of theforegoing described system, apparatus and unit, refer to thecorresponding processes in the foregoing method embodiments, and detailsare not described herein again.

In the several embodiments provided in this application, it is to beunderstood that the disclosed system, apparatus, and method may beimplemented in other manners. For example, the described apparatusembodiment is merely an example. For example, the unit division ismerely logical function division and may be other division during actualimplementation. For example, a plurality of units or components may becombined or integrated into another system, or some features may beignored or not performed. In addition, the displayed or discussed mutualcouplings or direct couplings or communication connections may beimplemented by using some interfaces. The indirect couplings orcommunication connections between the apparatuses or units may beimplemented in electronic, mechanical, or other forms.

The units described as separate components may or may not be physicallyseparated, and the components displayed as units may or may not bephysical units, and may be located in one place or may be distributedover a plurality of network units. Some or all of the units may beselected according to actual requirements to achieve the objectives ofthe solutions of the embodiments.

In addition, functional units in the embodiments of this application maybe integrated into one processing unit, or each of the units may bephysically separated, or two or more units may be integrated into oneunit. The integrated unit may be implemented in the form of hardware, ormay be implemented in the form of a software functional unit.

When the integrated unit is implemented in the form of a softwarefunctional unit and sold or used as an independent product, theintegrated unit may be stored in a non-transitory computer-readablestorage medium. Based on such an understanding, the technical solutionsof this application essentially, or the part contributing to the relatedart, or all or some of the technical solutions may be implemented in theform of a software product. The computer software product is stored in astorage medium and includes several instructions for instructing acomputer device (which may be a personal computer, a server, a networkdevice, or the like) to perform all or some of the steps of the methodsdescribed in the embodiments of this application. The foregoing storagemedium includes: any medium that can store program code, such as a USBflash drive, a removable hard disk, a read-only memory (ROM), a randomaccess memory (RAM), a magnetic disk, or an optical disc. The foregoingembodiments are merely intended for describing the technical solutionsof this application, but not for limiting this application. Althoughthis application is described in detail with reference to the foregoingembodiments, a person of ordinary skill in the art understand that theymay still make modifications to the technical solutions described in theforegoing embodiments or make equivalent replacements to some technicalfeatures thereof, without departing from the spirit and scope of thetechnical solutions of the embodiments of this application.

INDUSTRIAL PRACTICABILITY

In the embodiments of this application, a pathology image set isobtained by a microscope, the pathology image set including at least ato-be-evaluated image and associated images, the associated images andthe to-be-evaluated image being consecutive frame images, then a firststatus corresponding to the to-be-evaluated image is determinedaccording to the pathology image set, the first status being used forindicating a motion change of the to-be-evaluated image, and a secondstatus corresponding to the to-be-evaluated image is determinedaccording to the pathology image set when the first status is a staticstate, the second status being used for indicating a change in imageclarity of the to-be-evaluated image. In this way, moving stateevaluation and image clarity state evaluation can be performed onacquired images, to determine image statuses of different images, and animage status of a pathology image often reflects an operation of a useron a microscope and a change in a field of view of an image in themicroscope, so that pathology images acquired by a camera of themicroscope can be screened according to image statuses and a task type,to assist in completing a task purpose, thereby reducing the difficultyin image processing and improving the efficiency of task processing.

Note that the various embodiments described above can be combined withany other embodiments described herein. The features and advantagesdescribed in the specification are not all inclusive and, in particular,many additional features and advantages will be apparent to one ofordinary skill in the art in view of the drawings, specification, andclaims. Moreover, it should be noted that the language used in thespecification has been principally selected for readability andinstructional purposes, and may not have been selected to delineate orcircumscribe the inventive subject matter.

As used herein, the term “unit” or “module” refers to a computer programor part of the computer program that has a predefined function and workstogether with other related parts to achieve a predefined goal and maybe all or partially implemented by using software, hardware (e.g.,processing circuitry and/or memory configured to perform the predefinedfunctions), or a combination thereof. Each unit or module can beimplemented using one or more processors (or processors and memory).Likewise, a processor (or processors and memory) can be used toimplement one or more modules or units. Moreover, each module or unitcan be part of an overall module that includes the functionalities ofthe module or unit. The division of the foregoing functional modules ismerely used as an example for description when the systems, devices, andapparatus provided in the foregoing embodiments performs imageacquisition and/or image processing. In practical application, theforegoing functions may be allocated to and completed by differentfunctional modules according to requirements, that is, an innerstructure of a device is divided into different functional modules toimplement all or a part of the functions described above.

What is claimed is:
 1. A method, comprising: acquiring a pathology imageset using a microscope, the pathology image set including at least ato-be-evaluated image and one or more associated images, wherein theto-be-evaluated image and the one or more associated images areconsecutive frames of images acquired using the microscope; determininga first status corresponding to the to-be-evaluated image according tothe pathology image set, the first status being used for indicating amotion change of the to-be-evaluated image during the acquisition andthe first status includes a plurality of predefined states; and inaccordance with a determination that the first status corresponds to astatic state of the plurality of predefined states, determining a secondstatus corresponding to the to-be-evaluated image according to thepathology image set, wherein the second status indicates a change inimage clarity of the to-be-evaluated image.
 2. The method according toclaim 1, wherein: the one or more associated images comprise a firstassociated image and a second associated image that is consecutive tothe first associated image, the first associated image is acquiredbefore the to-be-evaluated image and the second associated image isacquired before the first associated image; and determining the firststatus further comprises: obtaining a first similarity between theto-be-evaluated image and the first associated image; in accordance witha determination that the first similarity is greater than a similaritythreshold, obtaining a second similarity between the first associatedimage and the second associated image; in accordance with adetermination that the second similarity is greater than the similaritythreshold, determining that the first status is the static state; and inaccordance with a determination that the second similarity is less thanor equal to the similarity threshold, determining that the first statusis a moving state-to-static state transition.
 3. The method according toclaim 1, wherein the one or more associated images include a firstassociated image and a second associated image that is consecutive tothe first associated image, the first associated image is acquiredbefore the to-be-evaluated image and the second associated image isacquired before the first associated image; and determining the firststatus further comprises: obtaining a first similarity between theto-be-evaluated image and the first associated image; in accordance witha determination that the first similarity is less than or equal to asimilarity threshold, obtaining a second similarity between the firstassociated image and the second associated image; in accordance with adetermination that the second similarity is greater than the similaritythreshold, determining that the first status is a transition state fromthe static state to a moving state; and in accordance with adetermination that the second similarity is less than or equal to thesimilarity threshold, determining that the first status is the movingstate.
 4. The method according to claim 2, wherein obtaining the firstsimilarity comprises: determining a source region pathology image setaccording to the to-be-evaluated image, the source region pathologyimage set comprising M source region images, wherein M is an integergreater than 1; determining a target region pathology image setaccording to the first associated image, the target region pathologyimage set comprising M target region images and the M target regionimages have image sizes that are less than image sizes of the sourceregion images; extracting a first source region image from the sourceregion pathology image set; extracting a first target region image fromthe target region pathology image set; in accordance with adetermination that the first source region image and the first targetregion image are background images: extracting a second source regionimage from the source region pathology image set; extracting a secondtarget region image from the target region pathology image set; anddetecting whether the second source region image and the second targetregion image are background images; and in accordance with adetermination that at least one of the first source region image and thefirst target region image is not a background image: calculating asimilarity between the first source region image and the first targetregion image; and using the calculated similarity as the similaritybetween the to-be-evaluated image and the first associated image.
 5. Themethod according to claim 4, wherein detecting whether the second sourceregion image and the second target region image are background imagesfurther comprises: calculating a pixel value standard deviation of thesecond source region image; determining, when the pixel value standarddeviation of the second source region image is less than or equal to astandard deviation threshold, that the second source region image is abackground image; calculating a pixel value standard deviation of thesecond target region image; and determining, when the pixel valuestandard deviation of the second target region image is less than orequal to the standard deviation threshold, that the second target regionimage is a background image.
 6. The method according to claim 4, whereincalculating the similarity between the first source region image and thefirst target region image further comprises: obtaining an image matrixthrough calculation according to the first source region image and thefirst target region image, the image matrix comprising a plurality ofelements; and determining the similarity between the first source regionimage and the first target region image according to the image matrix,the similarity between the first source region image and the firsttarget region image being a maximum value of the elements in the imagematrix.
 7. The method according to claim 1, wherein the one or moreassociated images include a first associated image that is consecutiveto and acquired before the to-be-evaluated image, and determining thesecond status further comprises: obtaining an image clarity of theto-be-evaluated image and an image clarity of the first associatedimage; in accordance with a determination that the image clarities ofthe to-be-evaluated image and the image clarity of the first associatedimage meet a first preset condition, obtaining an image clarity of abenchmark image; and in accordance with a determination that the imageclarities of the benchmark image and the to-be-evaluated image meet asecond preset condition, determining that the second status is afocusing state, the focusing state including a first change in focusfrom clear to blurred or a second change in focus from blurred to clear.8. The method according to claim 7, further comprising: after obtainingthe image clarities of the to-be-evaluated image and the firstassociated image: in accordance with a determination that the imageclarities of the to-be-evaluated image and the image clarity of thefirst associated image do not meet the first preset condition, updatingthe benchmark image to the to-be-evaluated image; and in accordance witha determination that the image clarities of the benchmark image and theimage clarity of the to-be-evaluated image do not meet the second presetcondition, updating the image clarity of the benchmark image when theimage clarity of the benchmark image and the image clarity of theto-be-evaluated image do not meet the second preset condition; and afterthe determining that the second status is a focusing state, the methodfurther comprises: updating the benchmark image for the to-be-evaluatedimage.
 9. The method according to claim 7, further comprising: afterobtaining the image clarities of the to-be-evaluated image and the firstassociated image: determining whether a difference between the imageclarity of the to-be-evaluated image and the image clarity of the firstassociated image is greater than or equal to a first image claritythreshold; determining, when the difference between the image clarity ofthe to-be-evaluated image and the image clarity of the first associatedimage is greater than or equal to the first image clarity threshold,that the image clarity of the to-be-evaluated image and the imageclarity of the first associated image meet the first preset condition;determining whether a difference between the image clarity of thebenchmark image and the image clarity of the to-be-evaluated image isgreater than or equal to a second image clarity threshold when thedifference between the image clarity of the to-be-evaluated image andthe image clarity of the first associated image is less than the firstimage clarity threshold, the second image clarity threshold beinggreater than the first image clarity threshold; and determining, whenthe difference between the image clarity of the benchmark image and theimage clarity of the to-be-evaluated image is greater than or equal tothe second image clarity threshold, that the image clarity of thebenchmark image and the image clarity of the to-be-evaluated image meetthe second preset condition.
 10. The method according to claim 1,further comprising: after determining the first status: storing theto-be-evaluated image when the first status corresponds to a firsttransition state from a moving state to the static state; and afterdetermining the second status, storing the to-be-evaluated image whenthe second status is a focusing state that includes a change fromblurred to clear.
 11. The method according to claim 1, furthercomprising: after determining the first status: performing pathologicalanalysis on the to-be-evaluated image when the first status is of atransition state from a moving state to the static state; and afterdetermining the second status, performing pathological analysis on theto-be-evaluated image when the second status is a focusing state thatincludes a change from blurred to clear.
 12. The method according toclaim 1, further comprising: after determining the first status:transmitting the to-be-evaluated image when the first status is: (1) afirst transition state from a moving state to the static state, (2) asecond transition state from the static state to the moving state, or(3) the moving state; and after determining the second status,transmitting the to-be-evaluated image when the second status is afocusing state that includes a change from blurred to clear or includesa change from clear to blurred.
 13. A computer device, comprising: oneor more processors; and memory storing one or more programs, that, whenexecuted by the one or more processors, cause the one or more processorsto perform operations comprising: acquiring a pathology image set usinga microscope, the pathology image set including at least ato-be-evaluated image and one or more associated images, wherein theto-be-evaluated image and the one or more associated images areconsecutive frames of images acquired using the microscope; determininga first status corresponding to the to-be-evaluated image according tothe pathology image set, the first status being used for indicating amotion change of the to-be-evaluated image during the acquisition andthe first status includes a plurality of predefined states; and inaccordance with a determination that the first status corresponds to astatic state of the plurality of predefined states, determining a secondstatus corresponding to the to-be-evaluated image according to thepathology image set, wherein the second status indicates a change inimage clarity of the to-be-evaluated image.
 14. The computer deviceaccording to claim 13, wherein: the one or more associated imagescomprise a first associated image and a second associated image that isconsecutive to the first associated image, the first associated image isacquired before the to-be-evaluated image and the second associatedimage is acquired before the first associated image; and determining thefirst status further comprises: obtaining a first similarity between theto-be-evaluated image and the first associated image; in accordance witha determination that the first similarity is greater than a similaritythreshold, obtaining a second similarity between the first associatedimage and the second associated image; in accordance with adetermination that the second similarity is greater than the similaritythreshold, determining that the first status is the static state; and inaccordance with a determination that the second similarity is less thanor equal to the similarity threshold, determining that the first statusis a moving state-to-static state transition.
 15. The computer deviceaccording to claim 14, wherein obtaining the first similarity comprises:determining a source region pathology image set according to theto-be-evaluated image, the source region pathology image set comprisingM source region images, wherein M is an integer greater than 1;determining a target region pathology image set according to the firstassociated image, the target region pathology image set comprising Mtarget region images and the M target region images have image sizesthat are less than image sizes of the source region images; extracting afirst source region image from the source region pathology image set;extracting a first target region image from the target region pathologyimage set; in accordance with a determination that the first sourceregion image and the first target region image are background images:extracting a second source region image from the source region pathologyimage set; extracting a second target region image from the targetregion pathology image set; and detecting whether the second sourceregion image and the second target region image are background images;and in accordance with a determination that at least one of the firstsource region image and the first target region image is not abackground image: calculating a similarity between the first sourceregion image and the first target region image; and using the calculatedsimilarity as the similarity between the to-be-evaluated image and thefirst associated image.
 16. The computer device according to claim 13,wherein the one or more associated images include a first associatedimage and a second associated image that is consecutive to the firstassociated image, the first associated image is acquired before theto-be-evaluated image and the second associated image is acquired beforethe first associated image; and determining the first status furthercomprises: obtaining a first similarity between the to-be-evaluatedimage and the first associated image; in accordance with a determinationthat the first similarity is less than or equal to a similaritythreshold, obtaining a second similarity between the first associatedimage and the second associated image; in accordance with adetermination that the second similarity is greater than the similaritythreshold, determining that the first status is a transition state fromthe static state to a moving state; and in accordance with adetermination that the second similarity is less than or equal to thesimilarity threshold, determining that the first status is the movingstate.
 17. The computer device according to claim 16, wherein detectingwhether the second source region image and the second target regionimage are background images further comprises: calculating a pixel valuestandard deviation of the second source region image; determining, whenthe pixel value standard deviation of the second source region image isless than or equal to a standard deviation threshold, that the secondsource region image is a background image; calculating a pixel valuestandard deviation of the second target region image; and determining,when the pixel value standard deviation of the second target regionimage is less than or equal to the standard deviation threshold, thatthe second target region image is a background image.
 18. Anon-transitory computer readable storage medium storing instructionsthat, when executed by one or more processors of a computer device,cause the one or more processors to perform operations comprising:acquiring a pathology image set using a microscope, the pathology imageset including at least a to-be-evaluated image and one or moreassociated images, wherein the to-be-evaluated image and the one or moreassociated images are consecutive frames of images acquired using themicroscope; determining a first status corresponding to theto-be-evaluated image according to the pathology image set, the firststatus being used for indicating a motion change of the to-be-evaluatedimage during the acquisition and the first status includes a pluralityof predefined states; and in accordance with a determination that thefirst status corresponds to a static state of the plurality ofpredefined states, determining a second status corresponding to theto-be-evaluated image according to the pathology image set, wherein thesecond status indicates a change in image clarity of the to-be-evaluatedimage.
 19. The non-transitory computer readable storage medium accordingto claim 18, wherein: the one or more associated images comprise a firstassociated image and a second associated image that is consecutive tothe first associated image, the first associated image is acquiredbefore the to-be-evaluated image and the second associated image isacquired before the first associated image; and determining the firststatus further comprises: obtaining a first similarity between theto-be-evaluated image and the first associated image; in accordance witha determination that the first similarity is greater than a similaritythreshold, obtaining a second similarity between the first associatedimage and the second associated image; in accordance with adetermination that the second similarity is greater than the similaritythreshold, determining that the first status is the static state; and inaccordance with a determination that the second similarity is less thanor equal to the similarity threshold, determining that the first statusis a moving state-to-static state transition.
 20. The non-transitorycomputer readable storage medium according to claim 19, whereinobtaining the first similarity comprises: determining a source regionpathology image set according to the to-be-evaluated image, the sourceregion pathology image set comprising M source region images, wherein Mis an integer greater than 1; determining a target region pathologyimage set according to the first associated image, the target regionpathology image set comprising M target region images and the M targetregion images have image sizes that are less than image sizes of thesource region images; extracting a first source region image from thesource region pathology image set; extracting a first target regionimage from the target region pathology image set; in accordance with adetermination that the first source region image and the first targetregion image are background images: extracting a second source regionimage from the source region pathology image set; extracting a secondtarget region image from the target region pathology image set; anddetecting whether the second source region image and the second targetregion image are background images; and in accordance with adetermination that at least one of the first source region image and thefirst target region image is not a background image: calculating asimilarity between the first source region image and the first targetregion image; and using the calculated similarity as the similaritybetween the to-be-evaluated image and the first associated image.